On non-asymptotic bounds for estimation in generalized linear models with highly correlated design

We study an increment bound for the empirical process, indexed by linear combinations of highly correlated base functions. We use direct arguments, instead of the chaining technique. We moreover obtain bounds for an M-estimation problem inserting a convexity argument instead of the peeling device. Combining the two results leads to non-asymptotic bounds with explicit constants. Let us motivate our approach. In M-estimation, some empirical average indexed by a parameter is minimized. It is often also called empirical risk minimization. To study the theoretical properties of the thus obtained estimator, the theory of empirical processes has been a successful tool. Indeed, empirical process theory studies the convergence of averages to expectations, uniformly over some parameter set. Some of the techniques involved are the chaining technique (see e.g. [13]), in order to relate increments of the empirical process to the entropy of parameter space, and the peeling device (a terminology from [10]) which goes back to [1], which allows one to handle weighted empirical processes. Also the concentration inequalities (see e.g. [9]), which consider the concentration of the supremum of the empirical process around its mean, are extremely useful in M-estimation problems. A more recent trend is to derive non-asymptotic bounds for M-estimators. The papers [6] and [4] provide concentration inequalities with economical constants. This leads to good non-asymptotic bounds in certain cases [7]. Generally however, both the chaining technique and the peeling device may lead to large constants in the bounds. For an example, see the remark following (5). Our aim in this paper is simply to avoid the chaining technique and the peeling device. Our results should primarily be seen as non-trivial illustration that both techniques may be dispensable, leaving possible improvements for future research. In particular, we will at this stage not try to optimize the constants, i.e. we will make some arbitrary choices. Moreover, as we shall see, our bound for the increment